Ambulatory system to communicate visual projections

ABSTRACT

An ambulatory system to communicate visual projections. An embodiment of an apparatus for ambulatory communication includes: a propulsion system to enable the apparatus to fly, including to hover in place and to follow a user; a stereo camera to record an image of a user of the apparatus or a scene nearby the user of the apparatus; a transmitter to transmit video data generated by the stereo camera to a second apparatus via network for a communication with a remote user; a receiver to receive video data via the network from the remote user; and a video projection mechanism to project an image including the received video to the user.

TECHNICAL FIELD

Embodiments described herein generally relate to the field of electronicdevices and, more particularly, to an ambulatory system to communicatevisual projections.

BACKGROUND

As electronic communications advance, there are more opportunities tocommunicate with others in various settings using video and audio. Videoconferencing, video telephone communications, and similar video toolshave become more common and more affordable for users.

In video conferencing and other video communications, the tools forcommunications include the possibility of the user of mobile devices forcommunication, including the use of laptop computers and handhelddevices,

However, communications remain largely stationary during thecommunication operation. While a laptop computer, for example, is easilymoved between locations, it is much more difficult to relocate theapparatus if a communication is in operation.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments described here are illustrated by way of example, and not byway of limitation, in the figures of the accompanying drawings in whichlike reference numerals refer to similar elements.

FIG. 1 is an illustration of components of an unmanned aerial vehicle(UAV) according to an embodiment;

FIG. 2 illustrates basic interactions between a UAV, an end user, and awireless system according to an embodiment;

FIG. 3 illustrates a communication process between two end usersaccording to an embodiment;

FIG. 4 illustrates a system and process for transmission of a realisticand spatially ambulatory image of a user to another user according to anembodiment;

FIG. 5 illustrates a system and process for transmission of a realisticand spatially ambulatory image of a scene to a remote user according toan embodiment;

FIG. 6 is an illustration of projection of blended reality between usersusing UAVs and supporting software according to an embodiment;

FIG. 7 illustrates an embodiment of an autonomous device or systemaccording to an embodiment;

FIG. 8 illustrates an embodiment of a mobile device according to anembodiment; and

FIG. 9 is a flow chart to illustrate a process for providingcommunications according to an embodiment utilizing an autonomous deviceto send and receive visual and auditory projections.

DETAILED DESCRIPTION

Embodiments described herein are generally directed to an ambulatorysystem to communicate visual projections.

For the purposes of this description:

“Mobile device” means a smartphone, tablet computer, handheld computer,mobile Internet device, or other mobile apparatus that includesprocessing ability and communication ability.

“Unmanned aerial vehicle” or “UAV” means an apparatus that flies withouta human pilot. As used herein, a UAV is a small device, the devicegenerally being much smaller than a human user of the UAV.

In some embodiments, an ambulatory apparatus or system operates tocommunicate visual projections to provide multi-party, fully immersiveand interactive augmented reality experience. In some embodiments, theambulatory apparatus or system further operates to communicate auditoryprojections. In some embodiments, the apparatus or system utilizes aconvergence of several technologies to provide enhanced communications,the apparatus or system utilizing one or more autonomous devices torecord and project video for communications. In some embodiments, theapparatus further utilizes one or more autonomous devices to record andproject audio for communications.

Virtual reality and augmented reality systems are becoming graduallymore popular as the technology increases in quality. A conventionalusage model involves a wearable element such as goggles or a helmet toallow the system to project images onto the user's eyes. The technologyhas advanced to a state where projected images may be convincing intheir realism and usefulness. However, these systems generally are notvery mobile using conventional technology, with the exception ofaugmented reality apps for devices such as smartphones and computertablets. Google Glass is an example of existing technology that may beutilized to provide a virtual reality experience for a user.

However, a usage model for which there is no acceptable conventionalsolution is one in which two or more users, who are geographicallydispersed, want to have an immersive virtual reality experience wherethe users can appear to be near to each other, see each other, and movewith each other away from a location such as a conference room or deskeven though such users may be thousands of miles away from each other.

In an example, a first person may be present in Oregon and a secondperson may be present in Hong Kong. Currently the parties may see andtalk with each other using certain conventional technologies. Forexample, a PC web cam will provide video communications. The existingtechnology for such operations may include Cisco Telepresence. However,this type of technology is limited in that a system requires the usersto be in front of a video and audio recording and transmitting device.In essence, both parties in a communication are required to remain ineach party's local conference room, cubicle, or other similar locationif they wish to have a “shared” communication experience.

In some embodiments, an ambulatory apparatus or system enables two ormore users who are separated by significant distances, such as hundredsor thousands of miles, to walk together around, for example, an office,a home, a shopping mall, a beach, a purely virtual world, or a blendedreality world.

In example, a first person in a first location may commence aconversation with a friend or colleague in a second location. In someembodiments, the parties could have a meeting while going for a walk.The first person would be able to see the second person walking next tothe first person, and could look the second person in the eye whilethese individuals walked and talked together. In some embodiments, anapparatus or system provides the ability for the second person to seethe first person and walk with the first person wherever such person islocated. Further, the second person could virtually ride with the firstperson in an automobile or other form of transportation. In someembodiments, an apparatus or system would include software that iscapable of stitching the vision together to make it appear the imageappear to be seamless to each of the users.

In some embodiments, the parties to a communication could agree to meetvirtually in location at which neither is physically present. Forexample, two parties may meet virtually at the Grand Canyon, and walkaround the canyon, even though the first person is in Oregon and thesecond person is in Hong Kong. In this example, an apparatus or systemincludes software that operates to paint the background together foreach of the parties to the communication.

In general, in order to enable a fully immersive multi-user virtualenvironment without restrictions on user movement or location, a systemis capable of recording and transmitting video and audio from a realphysical place that is near each user of the system. In order for afirst person to view the second person and walk with the second person adevice is required to be near each of the users to take the user'spicture and record such users.

In conventional operations, a wearable element such as a headset andvirtual reality goggles is needed for virtual reality connection. Insome embodiments, an apparatus or system does not require wearableelements for communication. In some embodiments, an apparatus or systemprovides immersive virtual reality by utilizing elements to record andproject video and sound from a location that is near each user, wheresuch elements include the ability to move and track each user.

In some embodiments, an apparatus or system includes one or more smallautonomous devices that are capable of hovering, tracking a user,projecting and recording video and sound. While the description hereingenerally describes a single autonomous device for simplicity,embodiments are not limited to a single autonomous device, and mayinclude one or more autonomous devices. In some embodiments, the tasksfor ambulatory communications may be divided among one or moreautonomous devices.

The power of smartphones, including new phones utilizing processors suchas the Intel Medfield processor, is increasing to the point where excessprocessing capabilities can be made available for new usage models.Battery power is also improving, though at a slow pace than processingcapability. However, the extra battery power that is becoming availableaffords new components to be added to a smartphone.

In some embodiments, an apparatus or system utilizes laser projection toprovide an image to a user. Laser projection systems are decreasing insize and power consumption, including pico projectors or projectorsutilizing Texas Instruments DLP technology. Further, research hasprogressed on Virtual Retinal Displays (VRD), where laser images can beprojected directly onto the user's eye.

In some embodiments, an apparatus of system provides for soundprojection directed to a user without the sound being significantlyaudible to others. In some embodiments, an autonomous device providesfor discretely directing sound to the user of the apparatus or system.In an example, research has occurred on parametric arrays utilizingheterodyning for projecting “ultrasonic” sound, where such soundprojection provides for sound to be projected along a narrow beam suchthat only the targeted user would hear the sound.

Unmanned aerial vehicles (UAVs) have become significantly smaller andmore sophisticated since such devices first became available. Suchdevices include a device presented by Regina Dugan of DARPA, the devicebeing a robotic “hummingbird” that is capable of flying for severalminutes, and a device developed by a research team from the RoyalMelbourne Institute of Technology in which modified acommercially-available quadrocopter converted into an autonomous, flyingrunning partner for solo joggers.

In some embodiments, an autonomous device for an ambulatorycommunication system is:

(1) Small in size, with the ability to fly, including hovering in place.

(2) Capable of staying near the user by tracking and following the user;

(3) Capable of projecting an image directly onto the user's eye or eyesvia low power laser projection, where such technology may includedvirtual retinal display.

(4) Includes necessary software and hardware to stitch or merge virtualand real worlds together for the experience of the user.

In some embodiments, an autonomous device for an ambulatorycommunication system is further:

(5) Capable of projecting sound to the user in a narrow beam such thatthe intended user is the only person who is able to clearly hear theprojected sound, although small amounts of sound leakage may occur.

While the operations are directed to an autonomous device, in someembodiments such operations may be shared between multiple devices.Further, certain features may be handled by a mobile device that is inthe possession of or nearby the user and that is operating cooperativelywith the autonomous device.

In some embodiments, an autonomous device provides services enablingambulatory virtual reality experiences. In some embodiments, theautonomous device is an un-tethered apparatus that enables an ambulatoryfully immersive multi-party virtual reality experience by combining (a)a small form factor UAV capable of hovering, (b) software that canidentify and track a user, (c) virtual retinal display, (d) softwaremethods for command and control, and (e) UAV to UAV communicationsystems in implementations in which two or more UAVs are used withinproximity to a user. In some embodiments, optionally for blended realitythe autonomous device further includes: (f) software methods forstitching together disparate scenes and backgrounds.

In some embodiments, the autonomous device further includes (g)parametric array utilizing heterodyning for projecting “ultrasonic”sound.

Additionally, in some embodiments an apparatus or system allowscommunication in which: (a) no user headset for sound is required, (b)no user goggles, glasses or lenses are required for video, (c) thesystem would allow for the remote viewer to obtain and maintain limitedcontrol of movement of the local device through the interpretation ofthe remote viewer (user) movements.

In some embodiments, a system allows for a remote viewer to obtain andmaintain limited control of movement of the local autonomous device. Inan example, the system allows a first person in a remote location to beable to look at a second person in a local location and to look awayfrom second person, such as in response to the second person suggestingthat the first person look at something in the local environment thatthe second person has pointed out. In some operations, the detection ofthe movement of the user generates a command that is transmitted to theremote device, the command directing the remote device to change aposition of the remote device, change an alignment of the remote device,or both.

FIG. 1 is an illustration of components of a UAV according to anembodiment. In some embodiments, a system to provide ambulatorycommunication includes one or more UAVs to fly and operate as autonomousdevices. In this illustration, a UAV 110 is represented as a box forsimplicity, with the individual components to provide lift and controlbeing known. The UAV 110 may be in the form of helicopter, aquadrocopter (quad-rotor helicopter), or other form that allows forhovering in place, moving to follow or move around a user, and turningto align towards the user or towards another location. The UAV may bereferred to as a VR (Virtual Reality) UAV.

In some embodiments, a UAV includes the following components 120:

(a) A VRD (Virtual Retinal Display) video projection mechanism;

(b) A central processing unit (CPU) or other processor or processors andrelated components to operate the UAV;

(c) A battery or other power element to power the UAV; and

(d) A stereo capable camera to capture video for communications, wherethe stereo video allows for providing an illusion of depth for the videoimage received by another user.

In some embodiments, a UAV further includes the following component:

(e) A sound projection mechanism.

FIG. 2 illustrates basic interactions between a UAV, an end user, and awireless system according to an embodiment. In some embodiments, a UAV210, such as UAV 110 (a VR UAV) illustrated in FIG. 1, is incommunication with a local user 220 and with a wireless network, such ascommunication with a wireless tower 230, where the wireless towerprovides a link to a wireless network to provide communications betweenthe user 220 and a remote user, who is not shown in this illustration.The UAV 210 is illustrated as a single unit for simplicity inillustration, but embodiments are not limited to a single UAV and mayinclude multiple UAVs, including a group or constellation of UAVs thatare in communication with each other.

In some embodiments, the UAV 210 views the local user 220 or the localenvironment and generates a stereo video recording of the view. In someembodiments, the UAV 210 provides video to the local user, such as videoprojection via laser. In some embodiments, the UAV 210 provides audio tothe local user, such as a narrowly focused sound projection. In someembodiments, while the UAV 210 is illustrated as being airborne, the UAVmay land in resting position to save power if this can be done whilecontinuing to provide operations in the system.

In some embodiments, the UAV 210 is engaged upon the commencement of acommunication between the local user 220 and another user. In someembodiments, engagement of the UAV 210 may include the UAV 210 becomingairborne and hovering near the user 220. In some embodiments, the UAV210 may have a storage location in which the UAV is located when not inoperation. In some embodiments, the storage location may provide forcharging of the battery of the UAV 210 when the UAV is not in operation.

FIG. 3 illustrates a communication process between two end usersaccording to an embodiment. In some embodiments, a first user, User A320, is viewed by and provided visual images by a first UAV 310, while asecond user, User B 370, is viewed by and provided visual images by asecond UAV 360. In some embodiments, User A 320 is further providedaudio by first UAV 310, and User B 370 is further provided audio bysecond UAV 360. In some embodiments, UAV 310 and UAV 360 have wirelessaccess to the Internet 350 (or other network), such as, for example, theUAVs having cellular connections that result in connection to theInternet. In some embodiments, the wireless access to the Internet maybe via mobile devices in the possession of the users, wherein the mobiledevices operate cooperatively with the UAVs.

FIG. 4 illustrates a system and process for transmission of a realisticand spatially ambulatory image of a user to another user according to anembodiment. In some embodiments, first UAV 410 records images of User A420. In some embodiments, second UAV 460 operates to direct videoimages, and may also direct audio, to User B 470, who may be in a remotelocation far away from User A 420.

In some embodiments, a process for projecting a view of User A 420 toUser B 470. In some embodiments, the process includes:

(1) Stereo video of User A 420 is recorded by UAV 410, such as recordingvideo through two channels, referred to as channel 1 and channel 2.

(2) UAV 410 transmits the recorded video of User A 420 via a network ornetworks, which may include initial transmission via a cellular network,a wireless wide area network (WWAN), or other wireless network followedby data transmission via the Internet. In some embodiments, the videomay be transmitted via a mobile device of User A 420.

(3) UAV 460 receives the video transmission via a cellular network,WWAN, or other wireless network. In some embodiments, the video may bereceived via a mobile device of User B 470.

(4) UAV 460 projects the channel 1 and channel 2 video of User A 420 toUser B 470 via VRD technology, thus providing a three-dimensional imageof User A that is viewed by User B.

FIG. 5 illustrates a system and process for transmission of a realisticand spatially ambulatory image of a scene to a remote user according toan embodiment. In some embodiments, first UAV 510 records images of ascene that is near User A 520. In some embodiments, second UAV 560operates to direct video images to User B 570, who may be in a remotelocation far away from User A 520. In this example, the Eiffel Tower (orany other location near User A) could be projected to User B 570. Thisexample demonstrates the ability of User B to have limited control overUser A's UAV. For example, User A 520 may direct the view of User B 570to the scene, and User B 570 may control the direction of UAV 510 by,for example, turning towards the direction that User A 520 is facing,and thereby generating a command for UAV 510.

In some embodiments, a process for projecting a view of a scene nearUser A 520 to User B 570. In some embodiments, the process includes:

(1) Stereo video of the desired scene is recorded by UAV 510, such asrecording video through two channels, referred to as channel 1 andchannel 2. In some embodiments, UAV 510 is directed towards the scene bycommand that includes a command generated by User B 570, where User B570 has been given limited ambulatory control over USV 510 by User A520.

(2) UAV 510 transmits the recorded video of the scene via a network ornetworks, which may include initial transmission via a cellular network,a WWAN, or other wireless network followed by data transmission via theInternet. In some embodiments, the video may be transmitted via a mobiledevice of User A 520.

(3) UAV 560 receives the video transmission of the scene via a cellularnetwork, WWAN, or other wireless network. In some embodiments, the videomay be received via a mobile device of User B 570.

(4) UAV 560 projects the video of the scene to User B 570 via VRDtechnology. In some embodiments, if the command or action to direct UAV510 to be directed towards the scene includes User B 570 changing adirection that User B facing, then the UAV 560 has either moved followedthe change of direction of User B 570 or taken any other actionnecessary to maintain needed alignment with User B 570.

In some embodiments, to provide a fully immersive virtual realityexperience, a system provides:

(1) Where each user is in a different geographic location, the users areable to view each other as if they were standing, sitting or walkingnear each other.

(a) Camera viewing User A is operable to track User A.

(b) Camera viewing User A is operable to transmit the image of User Afor projection to User B.

(2) Each user has the ability to allow the other user to view remotescenes without compromising either viewer's ability to orient themselvesspatially. In other words, each user can share the video from the otheruser without the user mistakenly moving into a dangerous locationbecause of a loss of spatial orientation.

(a) In some embodiments, the maintenance of spatial orientation ismanaged through software. This is evolving technology that is currentlyused in automotive and robotic systems for identifying structures, othersolid objects, and a road.

(3) In some cases the remote viewer can extend the remote viewer'sexperience spatially from the local viewer by, for example, movingaround a corner. In that case the local viewer would lose “sight” of theremote viewer.

(a) In some embodiments, in this case the system recognizes that one“view” is the dominant view. In the case of the preceding example, UserA would have the dominant view, wherein the dominant view allows User Bto have limited control over User A's UAV.

In some embodiments, an apparatus, system, or method allows control of adevice (UAV) based on a conversation, context and interactions betweentwo or more people. In some embodiments, multi-party contextual hardwareor software includes access control based on the immediate interactionsbetween users. In some embodiments, limited access control of a UAV by aremote user is provided as follows:

(1) User A accepts a call on a device (wherein the call may be receivedvia smartphone or via VR UAV) from User B.

(2) User B needs access to a service (wherein such service could be, forexample, access to a camera or to the GPS service).

(3) The system on User A's device is made aware of the request foraccess from user B.

(4) The User A device grants permission for User B to obtain ambulatorycontrol of the User A device. In some embodiments, control is providedwhere it is consistent with the observed interactions between two users.

In some embodiments, a modification of the operations illustrated inFIGS. 4 and 5 may include the use of a background of a location in whichneither User A or User B is located, with the system thus generated ablended reality. FIG. 6 is an illustration of projection of blendedreality between users using UAVs and supporting software according to anembodiment. In this illustration, User A is present in a first location610 while User B is present in a second location 620. In someembodiments, a system provides an ambulatory communication whereby UserA and User B are provided with a virtual reality (VR) location whereneither of the users is present 630. In some embodiments, User A andUser B are provided with a blended reality wherein each user sees theother user in the context of the VR location. In some embodiments, thesoftware of the system provides for stitching together the images of theusers in the VR location such that the users can tour through thelocation together, even though neither of the users is physicallypresent in the VR location.

FIG. 7 illustrates an embodiment of an autonomous device or systemaccording to an embodiment. In some embodiments, the autonomous deviceor system is a UAV 700, which may be referred to as a VR UAV. In thisillustration, certain standard and well-known components that are notgermane to the present description are not shown. Elements shown asseparate elements may be combined, including, for example, a system onchip (SoC) combining multiple elements on a single chip.

In some embodiments, the UAV 700 may operate to receive video audio fromand provide video and audio to a local user 760. In some embodiments,the UAV 700 may operate cooperatively with a mobile device (MD) 762,such as mobile device 800 illustrated in FIG. 8, to provide ambulatorycommunication operations for the local user 760. In some embodiments,the UAV 700, while generally described as a single autonomous device,may include a group or constellation of UAVs, where the UAVs may sharethe operations or mechanisms described here.

In some embodiments, the UAV 700 includes propulsion system 740, wherethe propulsion system 740 may include, for example, one or more rotorsto provide propulsion and to allow the UAV 700 to become airborne, hoverin place, rotate about one or more axes, and move in any direction. Insome embodiments, the UAV 700 may be powered, propelled, and controlledby external sources, where such operation may include, for example,electronic magnets that lift and direct the UAV; microwave energy beingprovided to power the UAV; or other external forms of power, propulsion,or control.

Under some embodiments, the UAV 700 includes an interconnect or crossbar702 or other communication means for transmission of data. Theinterconnect 702 is illustrated as a single interconnect for simplicity,but may represent multiple different interconnects or buses and thecomponent connections to such interconnects may vary. The interconnect702 shown in FIG. 7 is an abstraction that represents any one or moreseparate physical buses, point-to-point connections, or both connectedby appropriate bridges, adapters, or controllers.

In some embodiments, the UAV 700 includes a stereo camera (or multiplecameras) 730 to enable the recording of stereo video of a local user orscene. In some embodiments, the UAV includes a laser video projectionmechanism 732, such as a VRD mechanism, to enable the projection ofstereo video received from a remote user (or other source) to the eyesof a local user.

In some embodiments, the UAV 700 includes a directional microphone 734to enable the recording of audio from the local user or sounds near thelocal user. In some embodiments, a system may in addition or insteadutilize a microphone of a mobile device 762 operating cooperatively withthe UAV 700 to record audio. In some embodiments, the UAV 700 includes asound projection mechanism 736, where the mechanism allows forprojecting narrowly focused audio to the local user 760.

The UAV 700 may include a processing means such as one or moreprocessors 704 coupled to the interconnect 702 for processinginformation. The processors 704 may comprise one or more physicalprocessors and one or more logical processors. In some embodiments, theprocessors operate to control the propulsion 740 of the UAV 700, as wellas operating to handle transmitted and received audio and video data.

In some embodiments, the UAV 700 includes one or more transmitters orreceivers 710 coupled to the interconnect 702. In some embodiments, theUAV 700 may include one or more antennas 712 for the transmission andreception of data via radio signals. In some embodiments, the data thatis transmitted and received includes audio and video data to betransmitted to a remote user and audio and video data received from aremote user. In some embodiments, the transmitters and receivers 710 mayprovide communication with, for example, a wireless network; with themobile device operating cooperatively with the UAV; or with other UAVswithin a group of UAVs.

In some embodiments, the UAV 700 further comprises a random accessmemory (RAM) or other dynamic storage device or element as a main memoryand other memory (including memory buffers) 706 for storing informationand instructions to be executed by the processors 704. RAM memoryincludes dynamic random access memory (DRAM), which requires refreshingof memory contents, and static random access memory (SRAM), which doesnot require refreshing contents, but at increased cost. DRAM memory mayinclude synchronous dynamic random access memory (SDRAM), which includesa clock signal to control signals, and extended data-out dynamic randomaccess memory (EDO DRAM). The UAV 700 may include one or morenon-volatile memory elements 708, including, for example, flash memory,for the storage of certain elements. The UAV 700 also may comprise aread only memory (ROM) 709 or other static storage device for storingstatic information and instructions for the processors 704.

The UAV 700 may also comprise a power source 720, which may include abattery, a solar cell, a fuel cell, a charged capacitor, or other systemor device for providing or generating power in an apparatus. The powerprovided by the power source 720 may be distributed as required toelements of the UAV 700, and will, for example, supply the powernecessary to support the propulsion of the UAV 700. In some embodiments,the power source 720 may receive external power, including for examplethe receipt of microwave energy to power the UAV.

FIG. 8 illustrates an embodiment of a mobile device according to anembodiment. In this illustration, certain standard and well-knowncomponents that are not germane to the present description are notshown. Elements shown as separate elements may be combined, including,for example, an SoC combining multiple elements on a single chip.

In some embodiments, the mobile device 800 may work cooperatively with aUAV 864, such as the UAV 700 illustrated in FIG. 7, to provideambulatory communication operations for a local user 860. In someembodiments, the mobile device 800 may operate to receive and transmit aportion or all of the video and audio data from and to the local user860.

Under some embodiments, the mobile device 800 includes an interconnector crossbar 802 or other communication means for transmission of data.The interconnect 802 is illustrated as a single interconnect forsimplicity, but may represent multiple different interconnects or busesand the component connections to such interconnects may vary. Theinterconnect 802 shown in FIG. 8 is an abstraction that represents anyone or more separate physical buses, point-to-point connections, or bothconnected by appropriate bridges, adapters, or controllers.

In some embodiments, the mobile device 800 includes a microphone andspeaker 822. In some embodiments, the microphone may be utilized toenable the recording of audio from the local user for ambulatorycommunications. In some embodiments, a system may in addition or insteadutilize a microphone of a UAV operating cooperatively with the mobiledevice 800 to record audio.

The mobile device 800 may include a processing means such as the one ormore processors 804 coupled to the interconnect 802 for processinginformation. The processors 804 may comprise one or more physicalprocessors and one or more logical processors.

In some embodiments, the mobile device 800 includes one or moretransmitters or receivers 816 coupled to the interconnect 802. In someembodiments, the mobile device 800 may include one or more antennas 820for the transmission and reception of data via radio signals. In someembodiments, the data that is transmitted and received includes audioand video data to be transmitted to a remote user via a network andaudio and video data received from a remote user and to be transmittedto the UAV 864.

In some embodiments, the mobile device 800 further comprises a randomaccess memory (RAM) or other dynamic storage device or element as a mainmemory and other memory (including memory buffers) 806 for storinginformation and instructions to be executed by the processors 804. Themobile device 800 may include one or more non-volatile memory elements808, including, for example, flash memory, for the storage of certainelements. The mobile device 800 also may comprise a read only memory(ROM) 810 or other static storage device for storing static informationand instructions for the processors 804, and data storage 812, such as asolid state drive, for the storage of data.

The mobile device 800 may also comprise a battery 826 or other powersource, which may include a solar cell, a fuel cell, a chargedcapacitor, or other system or device for providing or generating powerin an apparatus. The power provided by the battery 826 may bedistributed as required to elements of the mobile device 800.

In some embodiments, the mobile device 800 includes one or more inputdevices 822 for the input of data, including hard and soft buttons, ajoy stick, a mouse or other pointing device, voice command system, orgesture recognition system. In some embodiments, the mobile deviceincludes an output display 824, where the display 824 may include aliquid crystal display (LCD) or any other display technology, fordisplaying information or content to a user. In some environments, thedisplay 824 may include a touch-screen that is also utilized as at leasta part of an input device 822.

FIG. 9 is a flow chart to illustrate a process for providingcommunications according to an embodiment utilizing an autonomous deviceto send and receive visual and auditory projections. In someembodiments, upon commencing a communication between a local user and aremote user 900, such as communication that begins with a mobile device,such as a cellular telephone call, there is an initiation of anambulatory communication session between the local user and the remoteuser 902. An operation to commence the ambulatory communication sessionmay include a discovery operation to determine whether each side of thecommunication is capable of ambulatory communications.

In some embodiments, a local autonomous device, such as a local UAV (ormultiple UAVs) of the user is engaged 904. In some embodiments, theengagement of the local UAV includes the UAV becoming airborne if theUAV is not currently in flight, and the UAV moving in position inrelation to the user to perform communication operations. In someembodiments, the local UAV operates to hover and follow the local userif the local user 906. In some embodiments, the operation of anambulatory communication session includes granting limited control ofthe local UAV to the remote user 908.

In some embodiments, at any time upon the end of the communication, suchas by either the local user or the remote user taking action to end thecommunication 910, the local UAV will end operation 912, where endingoperation may involve return to a resting position or charging location.

In some embodiments, the local UAV operates to record a stereo view ofthe local user or of a scene, where the scene may be a scene near thelocal user 914. In some embodiments, local audio is also recorded 916,where the local audio may be recorded by the UAV or by a mobile deviceworking cooperatively with the UAV.

In some embodiments, audio and stereo video data is transmitted via anetwork to the remover user 918. The transmission may occur by awireless connection between the UAV and a wireless network, or by thetransfer of the data to a mobile device working cooperatively with theUAV, with the mobile device then transferring the data via the wirelessnetwork to the remote user.

In some embodiments, audio and stereo video data is received from theremote user via the wireless network 920. In some embodiments, the audioand stereo video may be received directly by the UAV, or may be receivedby a mobile device working cooperatively with the UAV. In someembodiments, the UAV operates to provide the received stereo video fromthe remote user via laser projection, such as a VRD system, to the localuser 922, and to provide the received audio from the remote user via anarrowly focused audio projection to the local user 924.

In some embodiments, because of the granting of limited control of theUAV to the remote user 908, upon a command from the remote user to movethe UAV 926, such as a command generated by the remote user changing adirection a view or providing a voice or gesture command, the positionor direction of view of the local UAV may be shifted 928.

In the description above, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the described embodiments. It will be apparent,however, to one skilled in the art that embodiments may be practicedwithout some of these specific details. In other instances, well-knownstructures and devices are shown in block diagram form. There may beintermediate structure between illustrated components. The componentsdescribed or illustrated herein may have additional inputs or outputsthat are not illustrated or described.

Various embodiments may include various processes. These processes maybe performed by hardware components or may be embodied in computerprogram or machine-executable instructions, which may be used to cause ageneral-purpose or special-purpose processor or logic circuitsprogrammed with the instructions to perform the processes.Alternatively, the processes may be performed by a combination ofhardware and software.

Portions of various embodiments may be provided as a computer programproduct, which may include a computer-readable medium having storedthereon computer program instructions, which may be used to program acomputer (or other electronic devices) for execution by one or moreprocessors to perform a process according to certain embodiments. Thecomputer-readable medium may include, but is not limited to, magneticdisks, optical disks, compact disk read-only memory (CD-ROM), andmagneto-optical disks, read-only memory (ROM), random access memory(RAM), erasable programmable read-only memory (EPROM),electrically-erasable programmable read-only memory (EEPROM), magnet oroptical cards, flash memory, or other type of computer-readable mediumsuitable for storing electronic instructions. Moreover, embodiments mayalso be downloaded as a computer program product, wherein the programmay be transferred from a remote computer to a requesting computer.

Many of the methods are described in their most basic form, butprocesses can be added to or deleted from any of the methods andinformation can be added or subtracted from any of the describedmessages without departing from the basic scope of the presentembodiments. It will be apparent to those skilled in the art that manyfurther modifications and adaptations can be made. The particularembodiments are not provided to limit the concept but to illustrate it.The scope of the embodiments is not to be determined by the specificexamples provided above but only by the claims below.

If it is said that an element “A” is coupled to or with element “B,”element A may be directly coupled to element B or be indirectly coupledthrough, for example, element C. When the specification or claims statethat a component, feature, structure, process, or characteristic A“causes” a component, feature, structure, process, or characteristic B,it means that “A” is at least a partial cause of “B” but that there mayalso be at least one other component, feature, structure, process, orcharacteristic that assists in causing “B.” If the specificationindicates that a component, feature, structure, process, orcharacteristic “may”, “might”, or “could” be included, that particularcomponent, feature, structure, process, or characteristic is notrequired to be included. If the specification or claim refers to “a” or“an” element, this does not mean there is only one of the describedelements.

An embodiment is an implementation or example of the present invention.Reference in the specification to “an embodiment,” “one embodiment,”“some embodiments,” or “other embodiments” means that a particularfeature, structure, or characteristic described in connection with theembodiments is included in at least some embodiments, but notnecessarily all embodiments. The various appearances of “an embodiment,”“one embodiment,” or “some embodiments” are not necessarily allreferring to the same embodiments. It should be appreciated that in theforegoing description of exemplary embodiments, various features aresometimes grouped together in a single embodiment, figure, ordescription thereof for the purpose of streamlining the disclosure andaiding in the understanding of one or more of the various novel aspects.This method of disclosure, however, is not to be interpreted asreflecting an intention that the claimed embodiments requires morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, novel aspects lie in less than all features ofa single foregoing disclosed embodiment. Thus, the claims are herebyexpressly incorporated into this description, with each claim standingon its own as a separate embodiment.

In some embodiments, an apparatus for ambulatory communication includesa propulsion system to enable the apparatus to fly, including hoveringin place and following a user; a stereo camera to record an image of auser of the apparatus or a scene nearby the user of the apparatus; atransmitter to transmit video data generated by the stereo camera to asecond apparatus via network for a communication with a remote user; areceiver to receive video data via the network from the remote user; anda video projection mechanism to project an image including the receivedvideo to the user.

In some embodiments, the image projected to the user includes an imageof the remote user.

In some embodiments, the apparatus is to operate cooperatively with amobile device. In some embodiments, the transmitter is to transmit thevideo data via the mobile device to the network. In some embodiments,the receiver is to receive video data and audio data via the mobiledevice from the network.

In some embodiments, the video projection mechanism is to project animage using a laser projection. In some embodiments, the videoprojection mechanism is a Virtual Retinal Display (VRD).

In some embodiments, the apparatus further includes an audio projectionmechanism to project audio data to the user, wherein the receiver isfurther to receive the audio data via the network, and wherein the audioprojection mechanism is to project audio to the user in a focusedmanner.

In some embodiments, a system for ambulatory communication includes afirst set of one or more unmanned aerial vehicles UAVs for operationwith a first user; and a second set of one or more UAVs for operationwith a second user. In some embodiments, the first set of UAVs and thesecond set of UAVs are operable to support a communication sessionbetween the first user and the second user, wherein each of the sets ofUAVs includes: a stereo camera to record an image of the user of theapparatus or a scene nearby the user of the apparatus, a transmitter totransmit video data generated by the stereo camera to the other UAV, areceiver to receive video data via the network from the other UAV, and avideo projection mechanism to project an image including the receivedvideo to the user of the UAV.

In some embodiments, the system further includes a first mobile deviceto operate cooperatively with the first set of UAVs and a second mobiledevice to operate cooperatively with the second set of UAVs. In someembodiments, each set of UAVs is to transmit data via the respectivemobile device, and wherein each set of UAVs is to receive data via therespective mobile device. In some embodiments, the first mobile deviceis operable to record audio from the first user and the second mobiledevice is operable to record audio from the second user.

In some embodiments, the system further includes a first mobile deviceto operate cooperatively with the first set of UAVs and a second mobiledevice to operate cooperatively with the second set of UAVs, wherein thefirst mobile device is operable to record audio from the first user andthe second mobile device is operable to record audio from the seconduser, and each set of UAVs transmits data via the respective mobiledevice, and wherein each set of UAVs receives data via the respectivemobile device.

In some embodiments, the video projection mechanism of the first orsecond set of UAVs projects a blended reality image, the blended realityimage including a first video image that has been recorded by the otherset of UAVs and a second video image that has not been recorded byeither the first or second set UAVs. In some embodiments, the secondvideo image is an image of a location in which neither the first usernor the second user is located.

In some embodiments, the receiver of each of the sets of UAVs is furtherto receive audio data via the network from the other set of UAVs,wherein each of the sets of UAVs further includes an audio projectionmechanism to project the audio data to the user of the set of UAVs.

In some embodiments, a set of one or more UAVs includes a plurality ofUAVs, and each of the plurality of UAVs is operable to communicate withone or more of the other UAVs of the plurality of UAVs.

In some embodiments, the receiver of each of the sets of UAVs is furtherto receive audio data via the network from the other set of UAVs,wherein each of the sets of UAVs further includes an audio projectionmechanism to project the audio data to the user of the set of UAVs.

A method including commencing a communication between a local user and aremote user; initiating an ambulatory communication session for thelocal user and the remote user; enabling at least one autonomous device,wherein each autonomous device is operable to hover and follow the localuser; recording by the at least one autonomous device a stereo image ofthe local user or a scene; and transmitting data for the recorded stereoto a remote device for the remote user. In some embodiments, the atleast one autonomous device is a UAV.

In some embodiments, the method further includes stereo video data fromthe remote device; projecting to the local user by the at least oneautonomous device an image including the received stereo video data; andprojecting to the local user by the at least one autonomous device audioincluding the received audio data. In some embodiments, the imageprojected to the local user is a blended image including a first imagerecorded by the remote user and a second image that is not recorded bythe second device.

In some embodiments, the method further includes recording audio of thelocal user; transmitting data for the recorded audio to the remote user;receiving audio data from the remote user; and projecting to the localuser by the at least one autonomous device audio including the receivedaudio data.

In some embodiments, the recording of the audio of the local user isperformed by the at least one autonomous device.

In some embodiments, the recording of the audio of the local user isperformed using a mobile device operated by the local user, the mobiledevice operating cooperatively with the at least one autonomous device.

In some embodiments, functions of the at least one autonomous device areprovided by a single autonomous device.

In some embodiments, functions of the at least one autonomous device areprovided by a plurality of autonomous devices.

In some embodiments, machine-readable storage includes machine-readableinstructions that, when executed, implement the method.

In some embodiments, a non-transitory computer-readable storage mediumincludes instructions for commencing a communication between a localuser and a remote user; initiating an ambulatory communication sessionfor the local user and the remote user; enabling at least one autonomousdevice, wherein each autonomous device is operable to hover and followthe local user; recording by the at least one autonomous device a stereoimage of the local user or a scene; and transmitting data for therecorded stereo image to the remote user. In some embodiments, the atleast one autonomous device is a UAV.

In some embodiments, the medium further includes instructions forreceiving stereo video data from the remote user; and projecting to thelocal user by the at least one autonomous device an image including thereceived stereo video data.

In some embodiments, the medium further includes instructions forrecording audio of the local user; transmitting data for the recordedaudio to the remote device; receiving audio data from the remote device;and projecting to the local user by the at least one autonomous deviceaudio including the received audio data.

In some embodiments, the medium further includes instructions forstitching together a first image recorded by the remote user and asecond image that is not recorded by the remote user, the blended imagebeing the image projected to the local user by the at least oneautonomous device. In some embodiments, the first image is an image ofthe remote user and the second image is an image of a location in whichthe local user and the remote user are not located.

In some embodiments, an apparatus includes means for commencing acommunication between a local user and a remote user; means forinitiating an ambulatory communication session for the local user andthe remote user; means for enabling at least one autonomous device,wherein each autonomous device is operable to hover and follow the localuser; means for recording by the at least one autonomous device a stereoimage of the local user or a scene; and means for transmitting data forthe recorded stereo image and the recorded audio to a remote device ofthe remote user. In some embodiments, the at least one autonomous deviceis a UAV.

In some embodiments, the apparatus further includes means for receivingvideo data from the remote device; and means for projecting to the localuser by the at least one autonomous device an image including thereceived stereo video data.

In some embodiments, the apparatus further includes means for receivingaudio data from the remote device; means for projecting to the localuser by the at least one autonomous device audio including the receivedaudio data.

In some embodiments, the apparatus further includes means for recordingaudio of the local user; and means for transmitting data for the audioto the remote device.

In some embodiments, the apparatus further includes means for stitchingtogether a first image recorded by the remote user and a second imagethat is not recorded by the remote user, the blended image being theimage projected to the local user by the at least one autonomous device.

What is claimed is:
 1. An apparatus for ambulatory communicationcomprising: a propulsion system to enable the apparatus to fly,including to hover in place and to follow a user; a stereo camera torecord an image of a user of the apparatus or a scene nearby the user ofthe apparatus; a transmitter to transmit video data generated by thestereo camera to a second apparatus via network for a communication witha remote user; a receiver to receive video data via the network from thesecond apparatus; and a video projection mechanism to project an imageincluding the received video from the apparatus directly onto an eye oreyes of the user using a laser projection; wherein the apparatus doesnot include a projection screen, and wherein the image is viewable bythe user without use of virtual reality goggles or other wearableelement.
 2. The apparatus of claim 1, wherein the image projecteddirectly onto the eye or eyes of the user includes an image of theremote user.
 3. The apparatus of claim 1, wherein the apparatus is tooperate cooperatively with a mobile device.
 4. The apparatus of claim 3,wherein the transmitter is to transmit the video data via the mobiledevice to the network and the receiver is to receive video data via themobile device from the network.
 5. The apparatus of claim 1, wherein thevideo projection mechanism is a Virtual Retinal Display (VRD).
 6. Theapparatus of claim 1, further comprising an audio projection mechanismto project audio data to the user, wherein the receiver is further toreceive the audio data via the network, and wherein the audio projectionmechanism is to project audio to the user utilizing heterodyning in afocused beam such that only the user is able to hear the audio.
 7. Asystem for ambulatory communication comprising: a first set of one ormore unmanned aerial vehicles (UAVs) for operation with a first user;and a second set of one or more UAVs for operation with a second user;wherein the first set of UAVs and the second set of UAVs are operable tosupport a communication session between the first user and the seconduser; wherein each of the sets of UAVs includes: a stereo camera torecord an image of the user of the apparatus or a scene nearby the userof the apparatus, a transmitter to transmit video data generated by thestereo camera to the other set of UAVs, a receiver to receive video datavia the network from the other set of UAVs, and a video projectionmechanism to project an image including the received video directly fromthe video projection mechanism onto an eye or eyes of the user of theset of UAVs using a laser projection; wherein the system does notinclude a projection screen, and wherein the image is viewable by theuser without use of virtual reality goggles or other wearable element.8. The system of claim 7, further comprising a first mobile device tooperate cooperatively with the first set of UAVs and a second mobiledevice to operate cooperatively with the second set of UAVs.
 9. Thesystem of claim 8, wherein each set of UAVs is to transmit data via therespective mobile device, and wherein each set of UAVs is to receivedata via the respective mobile device.
 10. The system of claim 8,wherein the first mobile device is operable to record audio from thefirst user and the second mobile device is operable to record audio fromthe second user.
 11. The system of claim 7, wherein the video projectionmechanism of the first or second set of UAVs is to project a blendedreality image directly onto the eye or eyes of the respective user, theblended reality image including a first video image that has beenrecorded by the other set of UAVs and a second video image that has notbeen recorded by either the first or second set of UAVs.
 12. The systemof claim 11, wherein the second video image is an image of a location inwhich neither the first user or the second user is located.
 13. Thesystem of claim 7, wherein the receiver of each of the sets of UAVs isfurther to receive audio data via the network from the other set ofUAVs, wherein each of the sets of UAVs further includes an audioprojection mechanism to project the audio data to the user of the set ofUAVs.
 14. A method comprising: commencing a communication between alocal user and a remote user; initiating an ambulatory communicationsession for the local user and the remote user; enabling at least oneautonomous device, wherein each autonomous device is operable to hoverand follow the local user; recording by the at least one autonomousdevice a stereo image of the local user or a scene; transmitting datafor the recorded stereo image to a remote device for the remote user;receiving stereo video data from the remote device; and projecting tothe local user by the at least one autonomous device an image includingprojecting the received stereo video data directly onto an eye or eyesof the local user using a laser projection; wherein the projection ofthe image does not include a projection screen, and wherein the image isviewable by the user without use of virtual reality goggles or otherwearable element.
 15. The method of claim 14, wherein the imageprojected directly onto the eye or eyes of the local user is a blendedimage including a first image recorded by the remote device and a secondimage that is not recorded by the remote device.
 16. The method of claim14, further comprising: recording audio of the local user; transmittingdata for the recorded audio to the remote user; receiving audio datafrom the remote user; and projecting to the local user by the at leastone autonomous device audio including the received audio data.
 17. Themethod of claim 16, wherein the recording of the audio of the local useris performed by the at least one autonomous device.
 18. The method ofclaim 16, wherein the recording of the audio of the local user isperformed using a mobile device operated by the local user, the mobiledevice to operate cooperatively with the at least one autonomous device.19. A non-transitory computer-readable storage medium having storedthereon data representing sequences of instructions that, when executedby a processor, cause the processor to perform operations comprising:commencing a communication between a local user and a remote user;initiating an ambulatory communication session for the local user andthe remote user; enabling at least one autonomous device, wherein eachautonomous device is operable to hover and follow the local user;recording by the at least one autonomous device a stereo image of thelocal user or a scene; transmitting data for the recorded stereo imageand the recorded audio to a remote device for the remote user; receivingstereo video data from the remote device; and projecting to the localuser by the at least one autonomous device an image including projectingthe received stereo video data directly onto an eye or eyes of the localuser using a laser projection; wherein the projection of the image doesnot include a projection screen, and wherein the image is viewable bythe user without use of virtual reality goggles or other wearableelement.
 20. The medium of claim 19, further comprising instructionsthat, when executed by the processor, cause the processor to performoperations comprising: recording audio of the local user; transmittingdata for the recorded audio to the remote device; receiving audio datafrom the remote device; and projecting to the local user by the at leastone autonomous device audio including the received audio data.
 21. Themedium of claim 19, further comprising instructions that, when executedby the processor, cause the processor to perform operations comprising:stitching together a first image recorded by the remote user and asecond image that is not recorded by the remote user, the blended imagebeing the image projected directly onto the eye or eyes of the localuser by the at least one autonomous device.
 22. The medium of claim 21,wherein the first image is an image of the remote user and the secondimage is an image of a location in which the local user and the remoteuser are not located.
 23. A system for ambulatory communicationcomprising: a first set of one or more unmanned aerial vehicles (UAVs)for operation with a first user; and a second set of one or more UAVsfor operation with a second user; wherein the first set of UAVs and thesecond set of UAVs are operable to support a communication sessionbetween the first user and the second user; wherein the first set of oneor more UAVs are subject to control for a service by the second userupon granting of permission for such control by the first user; andwherein each of the sets of UAVs includes: a stereo camera to record animage of the user of the apparatus or a scene nearby the user of theapparatus, a transmitter to transmit video data generated by the stereocamera to the other set of UAVs, a receiver to receive video data viathe network from the other set of UAVs, and a video projection mechanismto project an image including the received video directly from the videoprojection mechanism onto an eye or eyes of the user of the set of UAVsusing a laser projection; wherein the apparatus does not include aprojection screen, and wherein the image is viewable by the user withoutuse of virtual reality goggles or other wearable element.
 24. The systemof claim 23, further comprising a first mobile device to operatecooperatively with the first set of UAVs and a second mobile device tooperate cooperatively with the second set of UAVs.
 25. The system ofclaim 24, wherein each set of UAVs is to transmit data via therespective mobile device, and wherein each set of UAVs is to receivedata via the respective mobile device.
 26. The system of claim 24,wherein the first mobile device is operable to record audio from thefirst user and the second mobile device is operable to record audio fromthe second user.